Lower bounds for non-convex stochastic optimization

We lower bound the complexity of finding $$\epsilon $$ -stationary points (with gradient norm at most ) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased oracle with bounded variance, we prove that (in worst case) any algorithm requires least ^{-4}$$ find point. The is tight, and establishes descent minimax optimal in this model. more restrictive noisy estimates satisfy mean-squared smoothness property, ^{-3}$$ queries, establishing optimality recently proposed variance reduction techniques.